Crown identification device, identification method, program, and recording medium

ABSTRACT

The present invention provides a system for identifying individual crowns of individual fruit trees using aerial images. A crown identification device  40  of the present invention includes an identification criterion determination unit  41  and a crown identification unit  42 . The identification criterion determination unit  41  includes a first image acquisition section  411  that acquires a first aerial image including a plurality of individual fruit trees in a deciduous period in a fruit farm field, a skeleton extraction section  412  that processes the first aerial image to extract a whole crown skeleton including the plurality of individual fruit trees, a vertex extraction unit  413  that extracts vertexes of each crown skeleton corresponding to each individual fruit tree, and an identification criterion extraction section  414  that extracts a crown candidate region of a minimum polygonal shape including all the vertexes as an identification criterion for each individual fruit tree and extracts a centroid of the crown candidate region. The crown identification unit  42  includes a second image acquisition section  421  that acquires a second aerial image of the fruit tree farm field at the time of identifying a crown at the same scale as the first aerial image, a whole crown extraction section  422  that processes the second aerial image to extract a whole crown image including the plurality of individual fruit trees, and a crown identification section  423  that collates the crown candidate region and the centroid of the identification criterion with the whole crown image to identify a crown region of each individual fruit tree in the second aerial image.

TECHNICAL FIELD

The present invention relates to a crown identification device, anidentification method, a program, and a recording medium.

BACKGROUND ART

In the breeding of fruit trees, pruning is very important. By carryingout pruning, space for growth of the branches is secured and workabilityof bud thinning and fruit thinning is improved. In general, fruitfarmers walk in the field to check the state of the crown for eachindividual fruit tree and carry out pruning and cultivation managementas necessary. However, the larger the field, the more labor and cost arerequired to walk the field and check the crown.

Therefore, in recent years, there has been proposed a method ofobtaining an aerial image of the field using drone or the like to checkthe state of the crown of each fruit tree (Non Patent Literatures 1 to6). If the state of the individual crowns of individual fruit trees canbe analyzed on the basis of the aerial image, it becomes possible toprune and cultivate the fruit trees more easily by referring to theinformation. However, crowning of fruit trees is complicated to optimizeproduction of unit area, and it is difficult to extract individualcrowns by the reported calculation method because the crowns ofindividual fruit trees are overlapped in the aerial image in a statewhere leaves grow on fruit trees.

CITATION LIST Patent Literature Non Patent Literature

-   Non Patent literature 1: Patrick A, Li C. High Throughput    Phenotyping of Blueberry Bush Morphological Traits Using Unmanned    Aerial Systems. Remote Sens 2017; 9: 1250.-   Non Patent literature 2: Diaz-Varela R A, de la Rosa R, Leon L,    Zarco-Tejada P J. High-Resolution Airborne UAV Imagery to Assess    Olive Tree Crown Parameters Using 3D Photo Reconstruction:    Application in Breeding Trials. Remote Sens 2015; 7: 4213-4232.-   Non Patent literature 3: Panagiotidis D, Abdollahnejad A, Surovy P,    Chiteculo V. Determining tree height and crown diameter from    high-resolution UAV imagery. Int J Remote Sens 2016; 1-19.-   Non Patent literature 4: Zarco-Tejada P J, Diaz-Varela R, Angileri    V, Loudjani P. Tree height quantification using very high resolution    imagery acquired from an unmanned aerial vehicle (UAV) and automatic    3D photo-reconstruction methods. Eur J Agron 2014; 55: 89-99.-   Non Patent literature 5: Shi Y, Thomasson J A, Murray S C et al.    Unmanned Aerial Vehicles for High-Throughput Phenotyping and    Agronomic Research. PLOS ONE 2016; 11: e0159781.-   Non Patent literature 6: Dunford R, Michel K, Gagnage M, Piegay H,    Tremelo M-L. Potential and constraints of Unmanned Aerial Vehicle    technology for the characterization of Mediterranean riparian    forest. Int J Remote Sens 2009; 30: 4915-4935.

SUMMARY OF INVENTION Technical Problem

It is therefore an object of the present invention to provide a newsystem for identifying individual crowns of individual fruit trees usingaerial images of a field, for example.

Solution to Problem

In order to achieve the aforementioned object, the present inventionprovides an identification device for identifying a crown of anindividual fruit tree in an image, the identification device including:an identification criterion determination unit and a crownidentification unit; the identification criterion determination unitincluding a first image acquisition section that acquires a first aerialimage including a plurality of individual fruit trees in a deciduousperiod in a fruit farm field, a skeleton extraction section thatprocesses the first aerial image to extract a whole crown skeletonincluding the plurality of individual fruit trees, a vertex extractionunit that extracts vertexes of each crown skeleton corresponding to eachindividual fruit tree, and an identification criterion extractionsection that extracts a crown candidate region of a minimum polygonalshape including all the vertexes as an identification criterion for eachindividual fruit tree and extracts a centroid of the crown candidateregion, the crown identification unit including a second imageacquisition section that acquires a second aerial image of the fruittree farm field at the time of identifying a crown at the same scale asthe first aerial image, a whole crown extraction section that processesthe second aerial image to extract a whole crown image including theplurality of individual fruit trees, and a crown identification sectionthat collates the crown candidate region and the centroid of theidentification criterion with the whole crown image to identify a crownregion of each individual fruit tree in the second aerial image.

The present invention also provides an identification method foridentifying crowns of a plurality of individual fruit trees in an image,the identification method including: an identification criteriondetermination step and a crown identification step; the identificationcriterion determination step including: a first image acquisition stepof acquiring a first aerial image including a plurality of individualfruit trees in a deciduous period in a fruit farm field, a skeletonextraction step of processing the first aerial image to extract a wholecrown skeleton including the plurality of individual fruit trees, avertex extraction step of extracting vertexes of each crown skeletoncorresponding to each individual fruit tree; and an identificationcriterion extraction step of extracting a crown candidate region of aminimum polygonal shape including all the vertexes as an identificationcriterion for each individual fruit tree and extracting a centroid ofthe crown candidate region, the crown identification step including: asecond image acquisition step of acquiring a second aerial image of thefruit tree farm field at the time of identifying a crown at the samescale as the first aerial image; a whole crown extraction step ofprocessing the second aerial image to extract a whole crown imageincluding the plurality of individual fruit trees, and a crownidentification step of collating the crown candidate region and thecentroid of the identification criterion with the whole crown image toidentify a crown region of each individual fruit tree in the secondaerial image.

The present invention also provides a program for a computer to executethe identification method according to the present invention.

The present invention also provides a computer readable recording mediumwith the program according to the present invention.

Advantageous Effects of Invention

According to the present invention, for example, by utilizing the aerialimage during the deciduous period, it is possible to easily identifycrowns of individual fruit trees with better accuracy. For this reason,the present invention is extremely useful for the cultivation of fruittrees in orchards.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example of an identificationdevice of the first example embodiment.

FIG. 2 is a block diagram illustrating an example of the hardwareconfiguration of the identification device of the first exampleembodiment.

FIGS. 3A to 3C are examples of images in the first example embodiment.

FIGS. 4A and 4B are examples of images in the first example embodiment.

FIGS. 5A and 5B are examples of images in the first example embodiment.

FIGS. 6A to 6C are examples of images in the first example embodiment.

FIGS. 7A and 7B are examples of images in the first example embodiment.

FIGS. 8A to 8D are examples of images in the first example embodiment.

FIG. 9 is a flowchart illustrating an example of the identificationmethod of the first example embodiment.

FIG. 10 is a block diagram illustrating an example of a breeding datacollection device of the third example embodiment.

FIG. 11 is a block diagram illustrating another example of the breedingdata collection device of the third example embodiment.

FIG. 12 is a block diagram illustrating yet another example of thebreeding data collection device of the third example embodiment.

FIG. 13 is a block diagram illustrating an example of an imageprocessing unit in the breeding data collection device of the thirdexample embodiment.

FIG. 14 is a flowchart illustrating an example of a method of the thirdexample embodiment.

FIG. 15 is a block diagram illustrating an example of the breeding datacollection device of the fourth example embodiment.

FIGS. 16A and 16B are schematic views illustrating an example of asystem of the fifth example embodiment.

FIG. 17 is a block diagram illustrating an example of the hardwareconfiguration of the system of the third example embodiment.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention will be described with reference tothe drawings. Note here that the present invention is not limited to thefollowing example embodiments. In the drawings, identical parts aredenoted by identical reference numerals. Each example embodiment can bedescribed with reference to the descriptions of other exampleembodiments, unless otherwise specified, and the configurations of theexample embodiments may be combined, unless otherwise specified.

First Example Embodiment

FIG. 1 is a block diagram illustrating the configuration of an exampleof the identification device 40 of the present example embodiment. Theidentification device 40 includes an identification criteriondetermination unit 41 and a crown identification unit 42. Theidentification criterion determination unit 41 includes a first imageacquisition section 411, a skeleton extraction section 412, a vertexextraction section 413, and an identification criterion extractionsection 414. The crown identification unit 42 includes a second imageacquisition section 421, a whole crown extraction section 422, and acrown identification section 423. The identification device 40 mayfurther include, for example, a fruit tree trait analysis unit 43 and astorage 44. The identification device 40 is also referred to as anidentification system, for example.

The identification device 40 may be, for example, one presentationdevice including the components, or may be a presentation device towhich the components are connectable via a communication network. Thecommunication network is not particularly limited, and a known networkcan be used. The communication network may be, for example, a wirednetwork or a wireless network. Examples of the communication networkinclude an Internet line, a telephone line, and a local area network(LAN), and a wireless fidelity (WiFi).

The identification device 40 may further include an output unit. Forexample, the identification device 40 includes, as the output unit, adisplay unit such as a display to be described later and may outputinformation obtained by the identification device 40 to the displayunit. The identification device 40 may include, for example, acommunication device to be described later as the output unit, and maybe connectable to an external device via a communication network. Inthis case, the identification device 40 may output information obtainedby the identification device 40 to the external device via thecommunication network. The external device is not particularly limited,and examples thereof include imaging devices such as a camera andterminals such as a personal computer (PC), a mobile phone, a smartphone, and a tablet. The communication network is not particularlylimited and is the same as described above.

FIG. 2 is a block diagram illustrating an example of a hardwareconfiguration of the identification device 40. The identification device40 includes, for example, a central processing unit (CPU) 50 as aprocessor, a memory 51, a bus 52, an input device 53, a display 54, acommunication device 55, a storage device 56, and the like. Thecomponents of the identification device 40 are connected to each othervia the bus 52 by, for example, respective interfaces.

The CPU 50 serves to control the entire control of the identificationdevice 40. In the identification device 40, the CPU 50 executes, forexample, the program of the present invention and other programs, andreads and writes various pieces of information. Specifically, forexample, the CPU 50 of the identification device 40 functions as anidentification criterion determination unit 41, a crown identificationunit 42, and a fruit tree trait analysis unit 43.

The identification device 40 is connectable to a communication networkby the communication device 55 connected to the bus 52, for example, andis connectable to the external device via the communication network.

The memory 51 includes, for example, a main memory, and the main memoryis also referred to as a main storage device. When the CPU 50 performsprocessing, the memory 51 reads various kinds of operation programs 57such as the program of the present invention stored in an auxiliarystorage device to be described later, and the CPU 50 receives data fromthe memory 51 and executes a program 57, for example. The main memoryis, for example, a random access memory (RAM). The memory 51 furtherincludes, for example, a read-only memory (ROM).

The storage device 56 is also referred to as, for example, an auxiliarystorage relative to the main memory (main storage). The storage device56 includes, for example, a storage medium and a drive for reading fromand writing to the storage medium. The storage medium is notparticularly limited and may be of, for example, a built-in type or anexternal type, and examples thereof include a hard disc (HD), a Floppy®card (FD), a CD-ROM, a CD-R, a CD-RW, an MO, a DVD, a flash memory, anda memory card. The drive is not particularly limited. The storage device56 may be, for example, a hard disk drive (HDD) in which the storagemedium and the drive are integrated. The storage device 56 stores, forexample, the program 57 as mentioned above, and when the CPU 50 isactuated, the memory 51 reads the program 57 from the storage device 56as mentioned above. The storage device 56 is, for example, the storage44, and stores information input to the identification device 40,information processed by the identification device 40, and the like.

The identification device 40 may further include, for example, an inputdevice 53 and a display 54. Examples of the input device 53 include atouch panel, a mouse, and a keyboard. Examples of the display 54 includean LED display and a liquid crystal display.

As described above, the identification criterion determination unit 41includes a first image acquisition section 411, a skeleton extractionsection 412, a vertex extraction section 413, and an identificationcriterion extraction section 414.

The first image acquisition section 411 acquires a first aerial image Fincluding a plurality of individual fruit trees in a deciduous period ina fruit farm field. The first image acquisition section 411 may acquirethe first aerial image F by inputting it using the input device 53 or byreceiving it from the external device via the communication network, forexample.

The first aerial image F is an image of a fruit tree farm field in adeciduous period and includes a plurality of individual fruit trees. Thedeciduous period is, for example, a period when leaves fall off from thebranches, and in the case of fruit trees, it is generally, for example,a winter period, specifically, around November to March in Japan. Theaerial image of the fruit tree farm field in the deciduous period canbe, for example, an aerial image of the fruit tree farm field whereleaves have fallen off. The first aerial image F can be obtained by anunmanned aerial vehicle such as a drone, for example.

The skeleton extraction section 412 processes the first aerial image Fto extract a whole crown skeleton F including a plurality of individualfruit trees. How to process images is not particularly limited. As aspecific example, this will be described with reference to FIGS. 3A to3C. A 3D image reflecting the heights of the fruit trees and the like asshown in FIG. 3B can be reconstructed from the aerial image as shown inFIG. 3A by referring to the imaging condition and the like, for example,and skeletons F of the crowns of the fruit trees can be extracted fromthis 3D image as shown in FIG. 3C. The crown generally means a portionof the tree above the ground, such as branches, leaves, stems, and thelike of the tree.

The crown skeletons F can be extracted, for example, using adaptivefiltering or image morphology processing.

The vertex extraction section 413 extracts vertexes of each crownskeleton corresponding to each individual fruit tree. The identificationcriterion extraction section 414 extracts a crown candidate region F ofa minimum polygonal shape including as an identification criterion andextracts a centroid F of the crown candidate region F for eachindividual fruit tree. The crown candidate region F is a region of aminimum polygonal shape including all the vertexes of the correspondingcrown skeleton and is specifically, for example, a region of a projectedminimum polygonal shape.

The extraction of the crown candidate region F will be described belowwith reference to FIGS. 4A and 4B as a specific example. The image ofthe crown skeletons F shown in FIG. 3C includes crown skeletons of 12individual fruit trees. As shown in FIG. 4A, the vertex extractionsection 413 extracts all vertexes of the corresponding crown skeleton ofeach individual fruit tree. Then, as shown in FIG. 4B, theidentification criterion extraction section 414 can extract a region(also referred to as a polygon) of the smallest polygonal shapesurrounded by a gray line including all the vertexes as each crowncandidate region F.

FIGS. 5A and 5B illustrate FIG. 4B in more detail. As indicated by thewhite line in FIG. 5A, the outer frame of the crown candidate region Fserving as an identification criterion can be extracted by the polygonof the minimum polygonal shape including all the vertexes. FIG. 5Billustrates the outer frame indicating the crown candidate region F inFIG. 5A as a black-and-white masked image.

Further, the identification criterion extraction section 414 extractsthe centroid F from the crown candidate region F. How to determine thecentroid F of the crown candidate region F is not particularly limited,and the centroid F can be determined by a commonly used method from themoment of the black-and-white masked image of FIG. 5B.

The crown identification unit 42 includes a second image acquisitionsection 421, a whole crown extraction section 422, and a crownidentification section 423.

The second image acquisition section 421 acquires a second aerial imageS of the fruit tree farm field at the time of identifying a crown at thesame scale as the first aerial image. The second aerial image S is notparticularly limited, and may be an image at the time when it isnecessary to identify crowns of fruit trees for training and pruning.The second aerial image S may be, for example, only an image (e.g., S1)at a specific time, or may be a plurality of images (e.g., S1, S2, S3, [. . . ], Sn, n is a positive integer) obtained over time at a pluralityof times.

In the present invention, a crown is identified using a crown candidateregion F extracted on the basis of the first aerial image F by theidentification criterion determination unit 41 and the centroid Fthereof. The second aerial image S is an image at the same scale as thefirst aerial image F. The second aerial image S may be brought to be atthe same scale as the first aerial image F by imaging under the samecondition as the first aerial image F or by image processing in advanceeven if imaged under different conditions, for example.

The whole crown extraction section 422 processes the second aerial imageS to extract a whole crown image S including the plurality of individualfruit trees. As described above, at the time when it is necessary toidentify the crown, the branches and leaves of the fruit trees overlapwith those of the adjacent fruit trees. Thus, it is difficult todistinguish the boundaries in the aerial image. The whole crownextraction section 422 processes the second aerial image S not for eachindividual fruit tree, but to extract a whole crown image S of theplurality of fruit trees.

How to process images is not particularly limited. A specific examplethereof will be described below with reference to FIGS. 6A to 6C. A 3Dimage reflecting the heights of the fruit trees and the like as shown inFIG. 6B can be reconstructed from the aerial image as shown in FIG. 6Aby referring to the imaging condition and the like, for example, and awhole crown image S of individual fruit trees can be extracted from this3D image as shown in FIG. 6C. The whole crown image S can be extracted,for example, using adaptive filtering or image morphology processing asdescribed above.

The crown identification section 423 collates the crown candidate regionF and the centroid F of the identification criterion with the wholecrown image S to identify a crown region S of each individual fruittree. As described above, in the first aerial image F, the crown regionF and the centroid thereof are extracted for each individual fruit treein the first aerial image F. The first aerial image F and the secondaerial image S are at the same scale. Therefore, by collating the crowncandidate region F as the identification criterion and the centroidthereof with the whole crown image S derived from the second aerialimage S, a crown region of each individual fruit tree in the secondaerial image S can be identified.

The crown identification section 423 may extract a crown candidateregion on the basis of the whole crown image S and may then identify thefinal crown region, for example. That is, first, the crown candidateregion and the centroid of the identification criterion may be collatedwith the whole crown image to extract a crown candidate region S of eachindividual fruit tree, and the crown region of each individual fruittree is then identified from the crown candidate region S and a minuteregion surrounding the crown candidate region S. The extraction of thecrown candidate region S on the basis of the crown region F and thecentroid thereof as the identification criterion and the identificationof the crown region S from the crown candidate region S will bedescribed in more detail below.

The whole crown image S is processed by a generic Watershedtransformation method, so that the tree crown image of each individualfruit tree is divided into a plurality of regions as shown in FIG. 7A.Then, using the crown candidate region F and the centroid F thereof asinitial values of the Watershed transformation method in the image ofFIG. 7A, the plurality of regions are integrated, and a crown candidateregion S is extracted as shown in FIG. 7B.

In each crown candidate region S shown in FIG. 8A (the same drawing asFIG. 7B), an uneven minute region S exists around the crown candidateregion S (for example, a region surrounded by a dotted line) as shown inFIG. 8B. Therefore, by calculating an Euclidean distance between thecrown candidate region S and the surrounding minute region S and, asshown in FIG. 8C, integrating the crown candidate region S and theminute region, a final crown region S can be identified as shown in FIG.8D.

As described above, the identification device 40 may further include afruit tree trait analysis unit 43 that analyzes an identified crownregion, for example. The trait of the crown region is, for example, acrown projected area, a crown diameter, a crown shape, or the like. Theinformation of these traits is obtained by statistical analysis, forexample, from each crown region in the image.

Next, the identification method of the present example embodiment willbe described with reference to the flowchart of FIG. 9 as an example.The identification method of the present example embodiment can beperformed using the identification device 40 of FIG. 1, for example. Theidentification method of the present example embodiment is not limitedto the use of the identification device 40 of FIG. 1.

First, the first image acquisition step, the skeleton extraction step,the vertex extraction step, and the identification criterion extractionstep are executed as an identification criterion determination step.

Specifically, in the first image acquisition step, the first imageacquisition section 411 acquires a first aerial image including aplurality of individual fruit trees in a deciduous period in a fruitfarm field (S10). Then, in the skeleton extraction step, the skeletonextraction section 412 processes the first aerial image to extract awhole crown skeleton including the plurality of individual fruit trees(S11). Further, in the vertex extraction step, the vertex extractionsection 413 extracts vertexes of each crown skeleton corresponding toeach individual fruit tree (S12). Next, in the identification criterionextraction step, the identification criterion extraction section 414extracts a crown candidate region of a minimum polygonal shape includingall the vertexes as an identification criterion for each individualfruit tree and extracts a centroid of the crown candidate region (S13).Then, the identification criterion extraction section 414 furtherextracts, as an identification criterion, a centroid of the crowncandidate region for each individual fruit tree (S14).

On the other hand, the second image acquisition step, the whole crownextraction step, and the crown identification step are executed as thecrown identification step.

Specifically, in the second image acquisition step, the second imageacquisition section 421 acquires a second aerial image of the fruit treefarm field at the time of identifying a crown at the same scale as thefirst aerial image (S15). Next, in the whole crown extraction step, thewhole crown extraction section 422 processes the second aerial image toextract a whole crown image including the plurality of individual fruittrees (S16).

Then, in the crown identification step, the crown identification section423 collates the crown candidate region and the centroid of theidentification criterion with the whole crown image to extract a crowncandidate region of each individual fruit tree (S17). This crowncandidate region may be identified as the final crown region.Alternatively, for example, the crown region may be identified from thecrown candidate region and the minute region as described above (S18).The crown region identified in this manner may be output (S20), or thetraits of the fruit tree may be analyzed on the basis of the identifiedcrown region (S19), and the analysis result may be output (S20).

The identification device and the identification method of the presentinvention can be utilized, for example, as image processing whenanalyzing a trait of a plant in the third example embodiment describedbelow.

Second Example Embodiment

The program of the present example embodiment can execute theidentification method of the above-described example embodiments on acomputer. The program of the present example embodiment may be recordedon, for example, a computer-readable recording medium. The recordingmedium is not particularly limited, and examples thereof include aread-only memory (ROM), a hard disk (HD), an optical disk, and a Floppy®disk (FD).

Third Example Embodiment

The identification device and the identification method for identifyinga crown of an individual fruit tree of the present invention can beutilized, for example, in the breeding data collection device and datacollection method of the present example embodiment.

In breeding studies, expression typing (phenotyping) of crops isgenerally performed by researchers entering a farm field, dividing thefarm field into sections (plots), and visually observing the plots.However, this method requires large labor for evaluation and large workcost, so that the measurement range and the measurement frequency arealso limited. Therefore, for example, phenotyping is not performed in afarm field where plants are actually bred, but is performed only in aspecial farm field for testing or the like.

In this regard, the breeding data collection device of the presentexample embodiment is a new system capable of easily obtaining data forphenotyping not only in a special farm field for testing, but also in afarm field where plants are actually bred, for example.

The breeding data collection device of the present example embodimentclassifies images of the farm field such as aerial images are classifiedon the basis of imaging altitudes reconstructs 3D image data of the farmfield, the sections in the farm field, and the plants in the farm fieldfrom the classified image groups, and further convert the 3D image datato visualization data based on the farm field condition or the imagingcondition, for example. Therefore, the present example embodiment canobtain, for example, visualization data of a wide area of an actual farmfield, rather than a limited area of a test farm field. As a result, thevisualization data can be widely used for the analysis of the actualfarm field and the plants grown there, and then can be used to, forexample, assist the acquisition of new knowledge on the phenotype.

FIG. 10 is a block diagram illustrating an example of the configurationof a breeding data collection device 1 of the present exampleembodiment. The breeding data collection device 1 includes a storage100, a processor 110, and an output unit 120. The processor 110 includesa classification unit 111, an image processing unit 112, and avisualization unit 113. The storage 100 includes an information storageunit 101 and a processed information storage unit 102.

The breeding data collection device 1 is also referred to as a breedingdata collection system, for example. The breeding data collection device1 may be, for example, one breeding data collection device including thecomponents, or a breeding data collection device in which the componentsare connectable to each other via a communication network. Thecommunication network is not particularly limited, and examples thereofare those described above.

FIG. 17 is a block diagram of a hardware configuration of the breedingdata collection device 1. The breeding data collection device 1includes, for example, a CPU 50 which is a processor, a memory 51, a bus52, an input device 53, a display 54, a communication device 55, and astorage device 56. The components of the breeding data collection device1 are connected to each other via the bus 52 by an interface (I/F), forexample.

The CPU 50 controls the breeding data collection device 1. In thebreeding data collection device 1, the CPU 50 executes a program of thepresent invention and other programs and reads and writes various piecesof information, for example. Specifically, in the breeding datacollection device 1, the CPU 50 functions as the processor 110, forexample.

For example, the breeding data collection device 1 is connectable to thecommunication network through the communication device 55 connected tothe bus 52 and is connectable to an external device via thecommunication network. When information is output to the externaldevice, the communication device 55 functions as the output unit 120,for example.

The memory 51 includes, for example, a main memory, and the main memoryis also referred to as a main storage device. When the CPU 50 executesprocessing, the memory 51 reads various kinds of operation programs 103such as the program of the present invention stored in an auxiliarystorage device to be described later, and the CPU 50 receives data fromthe memory 51 and executes the programs 103, for example. The mainmemory is, for example, the same as described above. The storage device56 is also referred to as, for example, an auxiliary storage relative tothe main memory (main storage).

The storage device 56 is, for example, the same as described above. Thestorage device 56 is, for example, a storage 100 and stores informationinput to the breeding data collection device 1, information processed inthe breeding data collection device 1, and the like, for example.

The breeding data collection device 1 further includes, for example, theinput device 53 and the display 54. Examples of the input device 53include a touch panel, a mouse, and a keyboard. Examples of the display54 include an LED display and a liquid crystal display. When informationis displayed in the breeding data collection device 1, the display 54functions as the output unit 120, for example.

In the breeding data collection device 1, information to be processed isinput from a client terminal 2 to the information storage unit 101 viaan interface (I/F) 31, as illustrated in FIG. 11, for example. The typeof the interface 31 is not particularly limited, and for example, agraphical user interface (GUI), a character user interface (CUI), anapplication program interface (API), or the like can be used.

The breeding data collection device 1 may be connected to the clientterminal 2 via the communication network 32, for example, as illustratedin FIG. 12. The communication network is not particularly limited, and aknown communication network can be used. The communication network maybe, for example, a wired network or a wireless network, and specificexamples thereof include an Internet line, a telephone line, and a localarea network (LAN).

As described above, the information storage unit 101 stores farm fieldinformation, an imaging condition including a flight log of aerialimaging, and aerial images of the farm field linked with the imagingcondition.

The farm field information is not particularly limited and examplesthereof include map information on the farm field, information on eachsection (plot) dividing the farm field, information on plants bred inthe farm field, and visual observation information. The information oneach section in the farm field is, for example, position information oneach section obtained by dividing the field into a plurality ofsections, specifically, coordinates and the like. The visual observationinformation is, for example, actual measurement information on plants inthe farm field.

The image of the farm field includes an aerial image of the farm field.As described above, when a researcher actually enters a farm field andobserves a plant, the researcher may perform ground imaging using acamera in addition to visual observation and use the ground image toevaluate the phenotype. However, when a researcher performs groundimaging of the farm field, the labor is large and the measurement rangeand the measurement frequency are limited, similarly to the visualobservation. In contrast, since the aerial image can be obtained by anunmanned aerial vehicle such as a drone, for example, an image of a widearea can be easily obtained regardless of the size of the farm field,and images over time can be easily obtained. Therefore, the presentexample embodiment can conduct high-speed phenotyping using the aerialimage. The image may further include, for example, a ground image.

The imaging condition includes an imaging altitude and further includes,for example, an imaging date and time. Further, as described above, forthe aerial image, the imaging condition includes, for example, a flightlog of an unmanned aerial vehicle that performs imaging. The flight logincludes, for example, a flight condition of the unmanned aerial vehicleused for imaging. The flight condition includes, for example, a flightroute, a flight speed, flight time, imaging date and time associatedwith flight, and an imaging time.

The classification unit 111 classifies, as mentioned above, the aerialimages into a plurality of image groups having different imagingaltitude ranges on the basis of an imaging altitude included in theimaging condition. By classifying the aerial images on the basis of theimaging altitude, suitable aerial images can be used, for example, forthe creation of image-processed data of the entire farm field, sectionsof the farm field, and plant groups in the farm field or the sections inthe processing by the image processing unit 112 to be described later.The imaging altitude range is not particularly limited and can be anyrange.

The number of image groups into which the aerial images are classifiedon the basis of the imaging altitude range is not particularly limited,and is preferably two or more. As a specific example, in the case ofclassifying into two image groups, for example, a first image group canbe set to an image group in which the imaging altitude is higher thanthat of a second image group. The imaging altitude of each image groupis not particularly limited, and for example, the following specificexample can be given. If the image-processed data for the entire farmfield is created by the image processing unit to be described later, thefirst image group includes, for example, aerial images, the imagingaltitude is, for example, about 100 m from the ground, and the imagingarea is about 5 ha. If the image-processed data for the plant groups ina section of the farm field is created by the image processing unit, thesecond image group includes, for example, aerial images, and the imagingaltitude is, for example, about 30 m from the ground, and the imagingarea is about 0.5 ha. In the case of the aerial imaging, for example,the imaging area can be calculated from the type of the image sensormounted on the unmanned aerial vehicle, the focal length, the verticalhovering accuracy of the unmanned aerial vehicle, and the like.

The image processing unit 112 first creates at least one image-processeddata selected from the group consisting of a two-dimensionalorthomosaic, a numerical surface model, and a point cloud from at leastone of the plurality of image groups and the imaging condition. Thetwo-dimensional orthomosaic is also referred to as, for example, anorthographic projection image. The numerical surface model is alsoreferred to as a digital surface model (DSM). The point cloud is alsoreferred to as, for example, a point group or three-dimensionalreconstruction data. The image-processed data can be used forvisualization of two-dimensional data, visualization ofthree-dimensional data (reconstruction into 3D image data), and the likeon the farm field, the sections of the farm field, or the plants in thefarm field, as will be described later.

Then, the image processing unit 112 analyzes traits of plants in thefarm field from the image-processed data to obtain trait analysis data.

The creation of the image processing and the analysis of the traitsusing the plurality of images included in the image group are notparticularly limited, and can be performed using, for example, existingsoftware. In the breeding data collection device 1 of the presentexample embodiment, for example, the software may be installed in theprocessor 110.

The image processing unit 112 may include, for example, a plurality ofimage processing units. As a specific example, FIG. 13 shows aconfiguration including a first image processing section 1121, a secondimage processing section 1122, and a third image processing section1123. In the present example embodiment, for example, the first imageprocessing section 1121 creates image-processed data on the entire farmfield (for example, reconstruction of a 3D image of the entire farmfield), and the second image processing section 1122 createsimage-processed data on sections of the farm field, specifically, aplant group in each section of the farm field (for example,reconstruction of a 3D image of each section of the farm field). Thethird image processing section 1123 can be used for, for example,creation of any image-processed data.

When the image processing unit 112 performs a plurality of kinds ofimage processing as shown in FIG. 13, for example, the kinds of imageprocessing may be performed by pipelining.

The first image processing section 1121 creates the image-processed datafrom a plurality of images (preferably aerial images) in the first imagegroup, for example. Specifically, for example, the plurality of imagesmay be aligned to reconstruct a 3D image of the entire farm field. Atthis time, for example, the 3D image of the entire farm field may bereconstructed by aligning and trimming the images on the basis of thefarm field information, the imaging information on the images, and thelike. The first image processing section 1121 may further analyze traitsof the plants in the entire farm field, for example, from theimage-processed data. The traits may be, for example, the plant coveragerate, the plant height, the plant growth rate, and the like in the farmfield and may include the condition of the farm field.

The second image processing section 1122 creates the image-processeddata from a plurality of images (preferably aerial images) in the secondimage group, for example.

Specifically, for example, the plurality of images may be aligned toreconstruct a 3D image of a section of the farm field. The second imageprocessing section 1122 may further analyze traits of the plants in thesection of the farm field, for example, from the image-processed data onthe basis of the farm field information, the imaging information on theimages, and the like. The traits may be the same as described above andare, for example, the plant coverage rate, the plant height, the plantgrowth rate, and the like in any area (any section).

The third image processing section 1123 creates any image-processed datafrom a plurality of images (preferably ground images) in any imagegroup, for example. Specifically, for example, a 3D image of plants inthe farm field can be reconstructed from the plurality of images.

The visualization unit 113 visualizes data obtained in the imageprocessing unit 112. The data obtained in the image processing unit 112is, for example, the image-processed data and trait analysis data on thetraits. The visualization unit 113 visualizes the data on the basis of,for example, at least one of the farm field condition or the imagingcondition. The 3D image data of the entire farm field can be linked withthe map data of the farm field to visualize or may be linked with animaging time to visualize, for example. The 3D image data of the entirefarm field and the map data of the farm field can be aligned by softwarerelating to geographic information such as QGIS, for example.Accordingly, the visualization can be performed in a time-sequentialmanner by being further linked with the imaging time. Hereinafter, thedata used in the visualization unit 113 is referred to as visualizationdata, and the data obtained in the visualization unit 113 is referred toas processed data.

The processed information storage unit 102 stores data obtained in theimage processing unit 112 (for example, image-processed data and traitanalysis data) and processed data obtained in the visualization unit113. The processed information storage unit 102 may further storeclassification data in the classification unit 111, for example.

The output unit 120 output at least one of the data obtained in theimage processing unit 112 or the data obtained in the visualization unit113. The output unit 120 may output the data to a client terminal or adisplay screen, for example. The output from the breeding datacollection device 1 to the outside may be output via an interface, forexample.

Next, a method for collecting breeding data (hereinafter also referredto as the breeding data collection method) of the present exampleembodiment is described below as an example with reference to theflowchart of FIG. 14.

The breeding data collection method of the present example embodiment isperformed as follows using the breeding data collection device 1 of thepresent example embodiment illustrated in FIG. 10, for example. Thebreeding data collection method of the present example embodiment is notlimited to the breeding data collection device 1 of FIG. 10.

First, in an information storage step (S1), farm field information, animaging condition including a flight log of aerial imaging, and aerialimages of the farm field linked with the imaging condition are stored.Next, in a classification step (S2), the aerial images are classifiedinto a plurality of image groups having different imaging altituderanges on the basis of an imaging altitude included in the imagingcondition.

Then, in an image processing step (S3), at least one image-processeddata selected from the group consisting of a two-dimensionalorthomosaic, a numerical surface model, and a point cloud is createdfrom at least one of the plurality of image groups and the imagingcondition, and a trait of a plant in the farm field is analyzed from theimage-processed data. In the present example embodiment, a case in whichthe images are classified into three image groups (first, second, andthird image groups) will be described as an example. In the first imageprocessing step (S3-1), a 3D image of the entire farm field isreconstructed from the first image group. In the second image processingstep (S3-2), a 3D image of a section of the farm field is reconstructedfrom the second image group to analyze traits of the plant group in thesection of the farm field. The third image processing step (S3-3) isoptional, and for example, an optionally selected image processing of anoptionally selected image group is performed.

Next, in the visualization step (S4), the obtained image-processed dataare visualized. Specifically, in a first visualization step (S4-1), theimage-processed data obtained in the first image processing step (S3-1)is visualized, in a second visualization step (S4-2), theimage-processed data obtained in the second image processing step (S3-2)is visualized, and in the third visualization step (S4-3), theimage-processed data obtained in the third image processing step (S3-3)is visualized.

Then, in a processed information storage step (S5), data obtained in theimage processing unit 112 and data obtained in the visualization unit113 are stored as processed information, and in an output step (S6), atleast one of the data obtained in the image processing unit 112 or thedata obtained in the visualization unit 113 is output.

The identification method of identifying a crown of an individual fruittree of the first example embodiment can be used from the acquisition ofan aerial image (for example, the step (S1)) to the image processing foranalyzing a trait (for example, the step (S3)) in the breeding datacollection method of the present example embodiment. In the presentexample embodiment, as described above, aerial images are acquired (step(S1)) and classified into image groups (step (S2)), the image-processeddata is created from the classified image groups, and traits of plantsin the farm field are analyzed from the image-processed data (step(S3)). Therefore, in the case in which the breeding data of the fruittrees in the fruit tree farm field is collected, for example, theimage-processed data is created (for example, the 3D image isreconstructed) from the aerial images of the fruit tree farm field.Then, as shown in the first example embodiment, for example, theextraction of skeletons to the extraction of the identificationcriterion are performed from this image-processed data of the firstimages. On the other hand, as shown in the first example embodiment, forexample, the extraction of the whole crown image to the identificationof the crown region are performed from this image-processed data of thesecond images. Accordingly, traits of plants (i.e., fruit trees) can beanalyzed as shown in the present example embodiment by analyzing thetraits of the identified crown region.

(Variation 1)

In the breeding data collection device 1 of the third exampleembodiment, for example, the information storage unit 101 may furtherstore sensing data of the farm field linked with position information onsensing in the farm field. The sensing data is not particularly limited,and examples thereof includes a temperature, a humidity, a carbondioxide level, and an amount of solar radiation (for example, aphotosynthetic photon flux density).

In the breeding data collection device 1, the visualization unit 113further visualizes the sensing data in the farm field linked with theposition information on the sensing on the basis of the farm fieldinformation. The data obtained in the visualization unit 113 is, forexample, graph data.

The present example embodiment can efficiently design a drone flightpath in imaging of the farm field, set an imaging condition of images,acquire phenotyping data, manage sensing data relating to an environmentof the farm field, and the like by using the processed data obtained inthe visualization unit 113, for example. Thus, for example, the presentinvention is also useful for high-speed phenotyping.

In addition, the breeding data collection device 1 of the presentexample embodiment can be a platform. Thus, the present invention can beintegrated with various other data, for example.

When a researcher directly collects data in the farm field, an objectiveor quantitative evaluate of the data may be difficult because it dependson the experience and intuition of researchers. In addition, the data iscollected by a researcher's own method, and it is difficult to share themethod. In contrast, the present example embodiment can bring images andthe like collected under individualistic conditions by variousresearchers to be stored in the information storage unit, for example.These images can be classified on the basis of the imaging altitude andthen can be processed and visualized. Therefore, not only informationobtained by individuals but also images collected under differentconditions by various researchers and processed information obtainedtherefrom can be accumulated, which leads to more efficient support fordiscovery of new findings.

Fourth Example Embodiment

FIG. 15 is a block diagram illustrating another example of the breedingdata collection device of the third example embodiment. In FIG. 15, abreeding data collection device 6 further includes a feature analysisunit 114. The feature analysis unit 114 is included, for example, in theprocessor 110. The breeding data collection device of the presentexample embodiment can also be referred to as, for example, a featureanalysis device or a feature analysis system.

The feature analysis unit 114 analyzes visualized data to extractfeatures of the farm field or the plant. As the visualization data, forexample, various data can be used, and specific examples thereof includethe feature amount of the image, created data, trait analysis data,sensing data, and the like. The extraction of the features is notparticularly limited, and can be extracted from outliers using, forexample, SVM or the like.

Fifth Example Embodiment

FIGS. 16A and 16B are schematic views illustrating another example ofthe breeding data collection device of the third example embodiment.FIG. 16A is an example of an overview of the breeding data collectiondevice 1 of the present example embodiment. FIG. 16B is an example of animage processing unit in the breeding data collection device 1. Thepresent invention is not limited thereto.

Sixth Example Embodiment

The program of the present example embodiment can execute the breedingdata collection method or the feature analysis method of the third orfourth example embodiment on a computer. The program of the presentexample embodiment may be recorded on, for example, a computer-readablerecording medium. The recording medium is not particularly limited, andexamples thereof include a read-only memory (ROM), a hard disk (HD), anoptical disk, and a Floppy® disk (FD).

While the present invention has been described above with reference toillustrative example embodiments, the present invention is by no meanslimited thereto. Various changes and variations that may become apparentto those skilled in the art may be made in the configuration andspecifics of the present invention without departing from the scope ofthe present invention.

This application claims priority from Japanese Patent Application No.2018-057034 filed on Mar. 23, 2018. The entire subject matter of theJapanese Patent Application is incorporated herein by reference.

(Supplementary Notes)

Some or all of the above example embodiments and examples may bedescribed as in the following Supplementary Notes, but are not limitedthereto.

(Supplementary Note A1)

An identification device for identifying a crown of an individual fruittree in an image, the identification device including:

an identification criterion determination unit; and

a crown identification unit,

the identification criterion determination unit including

-   -   a first image acquisition section that acquires a first aerial        image including a plurality of individual fruit trees in a        deciduous period in a fruit farm field,    -   a skeleton extraction section that processes the first aerial        image to extract a whole crown skeleton including the plurality        of individual fruit trees,    -   a vertex extraction unit that extracts vertexes of each crown        skeleton corresponding to each individual fruit tree, and    -   an identification criterion extraction section that extracts a        crown candidate region of a minimum polygonal shape including        all the vertexes as an identification criterion for each        individual fruit tree and extracts a centroid of the crown        candidate region,

the crown identification unit including

-   -   a second image acquisition section that acquires a second aerial        image of the fruit tree farm field at the time of identifying a        crown at the same scale as the first aerial image,    -   a whole crown extraction section that processes the second        aerial image to extract a whole crown image including the        plurality of individual fruit trees, and    -   a crown identification section that collates the crown candidate        region and the centroid of the identification criterion with the        whole crown image to identify a crown region of each individual        fruit tree in the second aerial image.

(Supplementary Note A2)

The identification device according to Supplementary Note A1, whereinthe crown identification unit collates the crown candidate region andthe centroid as the identification criterion with the whole crown imageto extract a crown candidate region of each individual fruit tree, andidentifies the crown region of each individual fruit tree from the crowncandidate region and a minute region surrounding the crown candidateregion.

(Supplementary Note A3)

The identification device according to Supplementary Note A1 or A2further including a fruit tree trait analysis unit that analyzes a traitof an identified crown region.

(Supplementary Note A4)

The identification device according to any one of Supplementary Notes A1to A3, further including an output unit that outputs the crown region.

(Supplementary Note A5)

An identification method for identifying crowns of a plurality ofindividual fruit trees in an image, the identification method including:

an identification criterion determination step; and

a crown identification step,

the identification criterion determination step including:

-   -   a first image acquisition step of acquiring a first aerial image        including a plurality of individual fruit trees in a deciduous        period in a fruit farm field,    -   a skeleton extraction step of processing the first aerial image        to extract a whole crown skeleton including the plurality of        individual fruit trees,    -   a vertex extraction step of extracting vertexes of each crown        skeleton corresponding to each individual fruit tree; and    -   an identification criterion extraction step of extracting a        crown candidate region of a minimum polygonal shape including        all the vertexes as an identification criterion for each        individual fruit tree and extracting a centroid of the crown        candidate region,

the crown identification step including:

-   -   a second image acquisition step of acquiring a second aerial        image of the fruit tree farm field at the time of identifying a        crown at the same scale as the first aerial image;    -   a whole crown extraction step of processing the second aerial        image to extract a whole crown image including the plurality of        individual fruit trees, and    -   a crown identification step of collating the crown candidate        region and the centroid of the identification criterion with the        whole crown image to identify a crown region of each individual        fruit tree in the second aerial image.

(Supplementary Note A6)

The identification method according to Supplementary Note A5, wherein

in the crown identification step,

the crown region and the centroid as the identification criterion arecollated with the whole crown image to identify a crown candidate regionof each individual fruit tree, and

the crown region of each individual fruit tree is identified from thecrown candidate region and a minute region surrounding the crowncandidate region.

(Supplementary Note A7)

The identification method according to Supplementary Note A5 or A6further including a fruit tree trait analysis step of analyzing a traitof an identified crown region.

(Supplementary Note A8)

The identification method according to any one of Supplementary Notes A5to A7, further including an output step of outputting the crown region.

(Supplementary Note A9)

A program for a computer to execute the identification method accordingto any one of Supplementary Notes A5 to A8.

(Supplementary Note A10)

A computer readable recording medium with the program according toSupplementary Note A9.

Some or all of the third to sixth example embodiments and examples maybe described as in the following Supplementary Notes, but are notlimited thereto.

(Supplementary Note B1)

A breeding data collection device for collecting breeding data in a farmfield, the breeding data collection device including:

an information storage unit that stores farm field information, animaging condition including a flight log of aerial imaging, and aerialimages of the farm field linked with the imaging condition;

a classification unit that classifies the aerial images into a pluralityof image groups having different imaging altitude ranges on the basis ofan imaging altitude included in the imaging condition;

an image processing unit that creates at least one image-processed dataselected from the group consisting of a two-dimensional orthomosaic, anumerical surface model, and a point cloud from at least one of theplurality of image groups and the imaging condition and analyzes a traitof a plant in the farm field from the image-processed data;

a visualization unit that visualizes the image-processed data obtainedin the image processing unit;

a processed information storage unit that stores, as processinginformation, the data obtained in the image processing unit and the dataobtained in the visualization unit; and

an output unit that outputs at least one of the data obtained in theimage processing unit or the data obtained in the visualization unit.

(Supplementary Note B2)

The breeding data collection device according to Supplementary Note B 1,wherein

the plurality of image groups at least includes a first image group anda second image group, and

the first image group has a higher imaging altitude range than thesecond image group.

(Supplementary Note B3)

The breeding data collection device according to Supplementary Note B1or B2, wherein the image processing unit reconstructs a 3D image of theentire farm field.

(Supplementary Note B4)

The breeding data collection device according to any one ofSupplementary Notes B1 to B3, wherein the image processing unitreconstructs a 3D image of a section of the farm field to determine agrowth condition of a plant group in the section.

(Supplementary Note B5)

The breeding data collection device according to any one ofSupplementary Notes B1 to B4, wherein the trait is at least one selectedfrom the group consisting of a plant coverage rate, a plant height, anda plant growth rate in the farm field.

(Supplementary Note B6)

The breeding data collection device according to any one ofSupplementary Notes B1 to B5, wherein the image processing unitreconstructs a 3D image of the plant to determine a trait of the plantin further detail.

(Supplementary Note B7)

The breeding data collection device according to any one ofSupplementary Notes B1 to B6, wherein

the image processing unit includes

a first image processing section that processes the first image group;and

a second image processing section that processes the second image group.

(Supplementary Note B8)

The breeding data collection device according to Supplementary Note B7,wherein the image processing unit executes a plurality of imageprocesses by pipelining.

(Supplementary Note B9)

The breeding data collection device according to any one ofSupplementary Notes B1 to B8, wherein

the visualization unit visualizes two-dimensional data andthree-dimensional data included in the data obtained in the imageprocessing unit.

(Supplementary Note B10)

The breeding data collection device according to Supplementary Note B9,wherein

the two-dimensional data includes data of a two-dimensional orthomosaic,a numerical surface model, and a graph, and

the three-dimensional data includes data of a point cloud.

(Supplementary Note B11)

The breeding data collection device according to any one ofSupplementary Notes B1 to B10, wherein

the imaging condition includes an imaging time, and

the visualization unit executes a visualization in a time-sequentialmanner.

(Supplementary Note B12)

The breeding data collection device according to any one ofSupplementary Notes B1 to B11, wherein

the information storage unit further stores sensing data of the farmfield linked with position information on sensing in the farm field.

(Supplementary Note B13)

The breeding data collection device according to Supplementary Note B12,wherein

the visualization unit further visualizes the sensing data in the farmfield linked with the position information on the sensing on the basisof the farm field information.

(Supplementary Note B14)

The breeding data collection device according to Supplementary Note B13,wherein the visualization unit obtains graph data.

(Supplementary Note B15)

The breeding data collection device according to Supplementary Note B13or B14, wherein the sensing data is at least one data selected from thegroup consisting of a temperature, a humidity, a carbon dioxide level,and an amount of solar radiation.

(Supplementary Note B16)

The breeding data collection device according to any one ofSupplementary Notes B1 to B15, wherein the farm field informationincludes coordinate information on each plot dividing the farm field andplant information on a plant growing in the farm field, and the plantinformation is linked with the coordinate information.

(Supplementary Note B17)

The breeding data collection device according to Supplementary Note B16,wherein the plant information is image analysis information on theplant.

(Supplementary Note B18)

The breeding data collection device according to any one ofSupplementary Notes B1 to B17, wherein the information storage unitfurther stores an imaging condition and a ground image linked with theimaging condition.

(Supplementary Note B19)

The breeding data collection device according to any one ofSupplementary Notes B1 to B18, wherein the aerial image is an imageobtained by an unmanned aerial vehicle.

(Supplementary Note B20)

The breeding data collection device according to Supplementary Note B19,wherein the unmanned aerial vehicle is a drone.

(Supplementary Note B21)

The breeding data collection device according to any one ofSupplementary Notes B1 to B20, further including:

a feature analysis unit that analyzes visualization data to extract afeature of the farm field or the plant, wherein

the visualization data is at least one selected from the groupconsisting of a feature quantity of each image, created data, traitanalysis data, and sensing data.

(Supplementary Note B22)

A breeding feature analysis device for analyzing a feature in breeding,the breeding feature analysis device being connectable to the breedingdata collection device according to any one of Supplementary Notes B1 toB21 via a communication network and including:

an input unit that inputs the visualization data of the device; and

a feature analysis unit that analyzes the visualization data andextracts a feature of the farm field or the plant.

(Supplementary Note B23)

A breeding data collection method for collecting breeding data of a farmfield, the breeding data collection method including:

an information storage step of storing farm field information, animaging condition including a flight log of aerial imaging, and aerialimages of the farm field linked with the imaging condition;

a classification step of classifying the aerial images into a pluralityof image groups having different imaging altitude ranges on the basis ofan imaging altitude included in the imaging condition;

an image processing step of creating at least one image-processed dataselected from the group consisting of a two-dimensional orthomosaic, anumerical surface model, and a point cloud from at least one image groupof the plurality of image groups and the imaging condition and analyzinga trait of a plant in the farm field from the image-processed data;

a visualization step of visualizing the image-processed data obtained inthe image processing step;

a processed information storage step of storing, as processinginformation, the data obtained in the image processing step and the dataobtained in the visualization step; and

an output step of outputting at least one of the data obtained in theimage processing step or the data obtained in the visualization step.

(Supplementary Note B24)

The breeding data collection method according to Supplementary Note B23,wherein

the plurality of image groups at least includes a first image group anda second image group, and

the first image group has a higher imaging altitude range than thesecond image group.

(Supplementary Note B25)

The breeding data collection method according to Supplementary Note B23or B24, wherein a 3D image of the entire farm field is reconstructed inthe image processing step.

(Supplementary Note B26)

The breeding data collection method according to any one ofSupplementary Notes B23 to B25, wherein a 3D image of a section of thefarm field is reconstructed to determine a growth condition of a plantgroup in the section in the image processing step.

(Supplementary Note B27)

The breeding data collection method according to any one ofSupplementary Notes B23 to B26, wherein the trait is at least oneselected from the group consisting of a plant coverage rate, a plantheight, and a plant growth rate in the farm field.

(Supplementary Note B28)

The breeding data collection method according to any one ofSupplementary Notes B23 to B27, wherein in the image processing step, a3D image of the plant is reconstructed to determine a growth conditionof the plant in further detail.

(Supplementary Note B29)

The breeding data collection method according to any one ofSupplementary Notes B23 to B28, wherein

the image processing step includes

-   -   a first image processing step of processing the first image        group; and    -   a second image processing step of processing the second image        group.

(Supplementary Note B30)

The breeding data collection method according to Supplementary Note B29,wherein in the image processing step, a plurality of image processes areexecuted by pipelining.

(Supplementary Note B31)

The breeding data collection method according to any one ofSupplementary Notes B23 to B30, wherein in the visualization step,two-dimensional data and three-dimensional data included in dataobtained in the image processing step are visualized.

(Supplementary Note B32)

The breeding data collection method according to Supplementary Note B31,wherein

the two-dimensional data includes data of a two-dimensional orthomosaic,a numerical surface model, and a graph, and

the three-dimensional data includes data of a point cloud.

(Supplementary Note B33)

The breeding data collection method according to any one ofSupplementary Notes B23 to B32, wherein

the imaging condition includes an imaging time, and

in the visualization step, a visualization is executed in atime-sequential manner.

(Supplementary Note B34)

The breeding data collection method according to any one ofSupplementary Notes B23 to B33, wherein in the information storage step,sensing data of the farm field linked with position information onsensing in the farm field is further stored.

(Supplementary Note B35)

The breeding data collection method according to Supplementary Note B34,wherein in the visualization step, the sensing data in the farm fieldlinked with the position information on the sensing is furthervisualized on the basis of the farm field information.

(Supplementary Note B36)

The breeding data collection method according to Supplementary Note B35,wherein data obtained in the visualization step is graph data.

(Supplementary Note B37)

The breeding data collection method according to Supplementary Note B35or B36, wherein the sensing data is at least one data selected from thegroup consisting of a temperature, a humidity, a carbon dioxide level,and an amount of solar radiation.

(Supplementary Note B38)

The breeding data collection method according to any one ofSupplementary Notes B23 to B37, wherein the farm field informationincludes coordinate information on each plot dividing the farm field andplant information on a plant growing in the farm field, and the plantinformation is linked with the coordinate information.

(Supplementary Note B39)

The breeding data collection method according to Supplementary Note B38,wherein the plant information is image analysis information on theplant.

(Supplementary Note B40)

The breeding data collection method according to any one ofSupplementary Notes B23 to B39, wherein in the information storage step,an imaging condition and a ground image linked with the imagingcondition are stored.

(Supplementary Note B41)

The breeding data collection method according to any one ofSupplementary Notes B23 to B40, wherein the aerial image is an imageobtained by an unmanned aerial vehicle.

(Supplementary Note B42)

The breeding data collection method according to Supplementary Note B41,wherein the unmanned aerial vehicle is a drone.

(Supplementary Note B43)

The breeding data collection method according to any one ofSupplementary Notes B23 to B42, further including:

a feature analysis step of analyzing visualization data to extract afeature of the farm field or the plant, wherein

the visualization data is at least one selected from the groupconsisting of a feature quantity of each image, created data, traitanalysis data, and sensing data.

(Supplementary Note B44)

A breeding feature analysis method for analyzing a feature in breeding,the breeding feature analysis method including:

a data collection step using the method according to any one ofSupplementary Notes B23 to B43; and

a feature analysis step of analyzing at least one visualization dataselected from the group consisting of a feature quantity of each image,the image-processed data, the trait analysis data, and sensing data toextract a feature of the farm field or the plant.

(Supplementary Note B45)

A program for a computer to execute the method according to any one ofSupplementary Notes B23 to B43.

(Supplementary Note B46)

A program for a computer to execute the method according toSupplementary Note B44.

(Supplementary Note B47)

A computer readable recording medium with the program according toSupplementary Note B45 or B46.

INDUSTRIAL APPLICABILITY

According to the present invention, for example, by utilizing the aerialimage during the deciduous period, it is possible to easily identifycrowns of individual fruit trees with better accuracy. For this reason,the present invention is extremely useful for the cultivation of fruittrees in orchards.

REFERENCE SIGNS LIST

-   40: Identification device-   41: Identification criterion determination unit-   411: First image acquisition section-   412: Skeleton extraction section-   413: Vertex extraction section-   414: Identification criterion extraction section-   42: Crown identification unit-   421: Second image acquisition section-   422: Whole crown extraction section-   423: Crown identification section-   43: Fruit tree trait analysis unit-   44: Storage-   1, 6: Breeding data collection device-   100: Storage-   101: Information storage unit-   102: Processed information storage unit-   110: Processor-   111: Classification unit-   112: Image processing unit-   113: Visualization unit-   120: Output unit-   2: Client terminal-   32: Communication network

1. An identification device for identifying a crown of an individualfruit tree in an image, the identification device comprising: anidentification criterion determination unit and a crown identificationunit; the identification criterion determination unit comprising a firstimage acquisition section that acquires a first aerial image including aplurality of individual fruit trees in a deciduous period in a fruitfarm field, a skeleton extraction section that processes the firstaerial image to extract a whole crown skeleton including the pluralityof individual fruit trees, a vertex extraction unit that extractsvertexes of each crown skeleton corresponding to each individual fruittree, and an identification criterion extraction section that extracts acrown candidate region of a minimum polygonal shape including all thevertexes as an identification criterion for each individual fruit treeand extracts a centroid of the crown candidate region, the crownidentification unit comprising a second image acquisition section thatacquires a second aerial image of the fruit tree farm field at the timeof identifying a crown at the same scale as the first aerial image, awhole crown extraction section that processes the second aerial image toextract a whole crown image including the plurality of individual fruittrees, and a crown identification section that collates the crowncandidate region and the centroid of the identification criterion withthe whole crown image to identify a crown region of each individualfruit tree in the second aerial image.
 2. The identification deviceaccording to claim 1, wherein the crown identification unit collates thecrown candidate region and the centroid as the identification criterionwith the whole crown image to extract a crown candidate region of eachindividual fruit tree, and identifies the crown region of eachindividual fruit tree from the crown candidate region and a minuteregion surrounding the crown candidate region.
 3. The identificationdevice according to claim 1 further comprising a fruit tree traitanalysis unit that analyzes a trait of an identified crown region. 4.The identification device according to claim 1, further comprising anoutput unit that outputs the crown region.
 5. An identification methodfor identifying crowns of a plurality of individual fruit trees in animage, the identification method comprising: an identification criteriondetermination step; and a crown identification step, the identificationcriterion determination step comprising: a first image acquisition stepof acquiring a first aerial image including a plurality of individualfruit trees in a deciduous period in a fruit farm field, a skeletonextraction step of processing the first aerial image to extract a wholecrown skeleton including the plurality of individual fruit trees, avertex extraction step of extracting vertexes of each crown skeletoncorresponding to each individual fruit tree; and an identificationcriterion extraction step of extracting a crown candidate region of aminimum polygonal shape including all the vertexes as an identificationcriterion for each individual fruit tree and extracting a centroid ofthe crown candidate region, the crown identification step comprising: asecond image acquisition step of acquiring a second aerial image of thefruit tree farm field at the time of identifying a crown at the samescale as the first aerial image; a whole crown extraction step ofprocessing the second aerial image to extract a whole crown imageincluding the plurality of individual fruit trees, and a crownidentification step of collating the crown candidate region and thecentroid of the identification criterion with the whole crown image toidentify a crown region of each individual fruit tree in the secondaerial image.
 6. The identification method according to claim 5, whereinin the crown identification step, the crown region and the centroid asthe identification criterion are collated with the whole crown image toidentify a crown candidate region of each individual fruit tree, and thecrown region of each individual fruit tree is identified from the crowncandidate region and a minute region surrounding the crown candidateregion.
 7. The identification method according to claim 5 furthercomprising a fruit tree trait analysis step of analyzing a trait of anidentified crown region.
 8. The identification method according to claim5, further comprising an output step of outputting the crown region. 9.(canceled)
 10. A non-transitory computer readable recording medium witha program for a computer to execute the identification method accordingto claim
 5. 11-57. (canceled)